DOCUMENT RESUME 



ED 398 259 



TM 025 212 



AUTHOR 
TITLE 
PUB DATE 
NOTE 

PUB TYPE 



EDRS PRICE 
DESCRIPTORS 



Kitao, S. Kathleen; Kitao, Kenj i 
Testing Writing. 

[96] 

7p. 

Guides ~ Non~Class room Use (055) Reports ~ 
Descriptive (141) 

MF01/PC01 Plus Postage. 

Difficulty Level; ^English (Second Language); Essays; 
Foreign Countries; Grammar; Language Proficiency; 
^Language Tests; Language Usage; Objective Tests; 
*Scoring; Second Language Learning; *Test 
Construction; Test Content; Timed Tests; Writing 
Evaluation; ^Writing Tests 



ABSTRACT 

Testing language skills is difficult, but testing 
writing, and the writing of students of English as a Second Language, 
poses two major problems. The first is making decisions about the 
matter of control, objectivity of the evaluation, and naturalness in 
the writing test. The second major problem is that, if the test is 
done in a way that cannot be graded objectively, it is necessary to 
develop a scale that makes grading as objective as possible. The 
ability to write involves grammatical and lexical abilities, 
mechanical ability, stylistic and organizational skills, and the 
ability to judge whether something is appropriate. Common tasks for 
writing tests include: (1) gap filling; (2) form completion; (3) 
making corrections; (4) letter writing; and (5) essay writing. Any 
chosen task should be evaluated for its relevance to the student's 
eventual use of the language. When assessing students at intermediate 
and advanced levels, test makers must consider the instructions, the 
choice of topics, the choice of tasks, and the level of difficulty 
and time allowed. All of these considerations must go into making a 
test that is appropriate for the learner, and then the test maker 
must attempt to ensure that marking the test, which will always be at 
least somewhat subjective, is as objective as possible. (SLD) 
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Testing each skill is uniquely difficult, but testing writing presents two 
particular problems. Testing writing presents two major problems. The 
first is making decisions about the matter of control, objectivity of the 
evaluation, and naturalness in the writing test. If you decide to test writing 
in a controlled way and in a way that can be graded objectively, you must 
do so in a way that does not necessarily reflect how the writing is used by 
the students in the real world. If, on the other hand, you test writing in a 
way that would reflect how the students use writing in the real world, it is 
difficult to have control over the writing and to evaluate the student’s work 
objectively. 

The second major problem with testing writing is, if the test is done in a 
way that it cannot be graded objectively, it is necessary to develop a scale 
that allows it to be graded as objectively as possible. How this is done is 
one of the great difficulties of testing writing. 

Components of Writing 
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The ability to write involves at least six component skills. They are: 

♦grammatical ability. This is the ability to write English in grammatically 
correct sentences. 

♦lexical ability. The ability to choose words that are correct and used 
appropriately. 

♦mechanical ability. The ability to correctly use punctuation, spelling, 
capitalizaton, etc. 

♦stylistic skills. The ability to use sentences and paragraphs appropriately. 

♦organizational skills. The ability to organize written work according to 
the conventions of English, including the order and selection of material. 

♦judgements of appropriacy. The ability to make judgements about what 
appropriate depending on the task, the purpose of the writing, and the 
audience. 
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Perhaps the most difficult--and most important--of these skills is the 
last. Native English speakers develop a sense of what is appropriate in 
different writing situations (though they may be taught to use specialized 
registers, such as academic English or business English). Registers of 
English range from very informal forms such as colloquialisms, slang, and 
jargon to standard English to more formal forms, such as the language 
used for business letters, legal documents, and academic papers. Writers 
must be aware of these differences and leam to follow the conventions of 
different situations. A writing test needs to take these skills into account. 

Types of Writing Tasks 



Gap Filling 

One of the most controlled way of testing writing is gap filling. Testees 
are presented with a passage with blanks, and they fill in the blanks. This 
is a mixture of both reading and writing skills, which is sometimes a 
problem, because it makes it difficult to decide what the scores really 
mean. However, with lower level students, it might be the only reasonable 
test of productive ability. 

Form Completion 

Another controlled way of testing writing is to have the testers fill oat a 
form, for example, an application. The advantage of such a task is that it 
is at least somewhat communicative, but the disadvantage is that it does 
not require any connected discourse or any use of language greater than 
lexical knowledge and a small amount of grammar. 

Making Corrections 

In some situations, testees are presented with a short piece of writing 
which has deliberate grammar, punctuation and spelling errors, and they 
are asked to correct the errors. While this task does related to one thing 
that people do when they write-editing-and it is objectively corrected, 
but it does not represent the writing task as a whole. 

Letter Writing 

Letter writing is a common task for writing tests. The stimulus for the 
letter may be a situation that is explained in the instructions, a letter to 
which the testees are instructed to respond, information given in chart or 
graph form that is to be summarized in the testees' letter, pictures or 
drawings that give information about a situation the testees are expected to 
write a letter about, etc. In all of these possibilities, the tester must keep in 
mind that the situation must be as clear as possible for the testee, unless 
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there is an intention to test reading and writing together. If the testee does 
not understand, for example, a letter that he/she is expected to respond to, 
it will be impossible to get a sample of writing to evaluate. 

Essay Writing 

Essay writing is probably one of the more common writing tasks, but it 
should be used carefully. If the future situation of the students will not 
include writing essays, the tester should carefully consider whether it is the 
best test of the students' writing ability. 

Issues Related to Intermediate and Advanced Writing Tests 

For beginners, it is often necessary to test writing indirectly, for 
example, by giving them very limited tasks that involve such activities as 
filling in blanks and which do not require them to write connected prose. 
However, once students get to the intermediate and advanced stages, their 
writing proficiency should be tested in direct ways, ways that allow 
students to demonstrate their ability to used connected discourse in 
appropriate ways, which is the way that they will use writing most in real 
life. The following is a list of desirable characteristics of such tests. For 
practical and other reasons, it may not be possible for a test to have all of 
these characteristics, but a teacher constructing a writing test should 
carefully consider the issues related to them. 

Instructions 

The instructions should use as few words as possible, consistent with 
making the writing task and situation clear, so that the testee does not 
spend too much time on the instructions. Definitions should be provided 
in the instructions, if any are necessary. 

Time and length. The instructions should also indicate the time 
allowed for the task and any instructions related to length in number of 
words. It may be useful to give the students a range of the number of 
words, so they have an idea of what is expected, though they should be 
encouraged not to spend too much time counting words. 

Establishing a context and communicative purpose. The test should 
establish a clear purpose of communicating, especially by indicating the 
intended reader and giving a realistic context for the reader. In making the 
context realistic, the teacher should keep in mind, for example, that the 
testee would probably not use English in writing to someone who shares 
his/her native language, so instructions to write a letter to a family member 



would not be an appropriate use of context. Establishing the context 
should be done in as few words as possible, while still making the 
situation clear. 

Choice of Topic(s) 

Choosing the topic is a very important part of the writing task. If the 
testees find that the topic is something that they cannot respond to, they 
will not be able to show their writing ability. There are various 
approaches to choosing a topic and various issues to consider. There are 
not always clear answers about what type of topic is best, but the teacher 
should be aware of the issues involved. 

Doability. The topic should be one that the testees are able to respond 
to. To determine this, it is necessary to pre-test the topic. A topic that 
sounds very reasonable in context may not work well in reality. 

Knowledge. The tester must consider the background knowledge that 
the students have about the topic. The topic should be one that the 
students have an approximately equal familiarity with. If students have 
different levels of knowledge about a topic, the ones with greater 
knowledge about it will have a clear advantage in writing about the topic. 
One approach is to choose a topic about which the testees are equally 
ignorant. However, this requires giving them enough information that they 
can write, which may result in the testing of the reading skill as well as 
writing, unless the information can be presented visually. Otherwise, a 
topic must be chosen that the students are about equally knowledgeable 
about. 

Motivation. The tester should consider the issue of motivation. Will 
the topic motivate students of the age, sex, field of study, background, etc., 
of the testees to write? It is often difficult to find a topic that will motivate 
all students equally. Some testers choose the strategy of choosing a 
subject that none of the students are likely to be motivated by. If none of 
the students are motivated, they will at least be on equal footing. The 
problem with this approach is that it may be difficult for students to do 
their best at showing their writing skill if they are not motivated by the 
topic. 

Breadth. The topic needs to be broad enough that every testee can 
approach it from some angle. If the topic is too narrow, the testees have 
little flexibility in their approach to it and may not have an opportunity to 
show their writing proficiency. 



Allowing students to choose topics. In some cases, students are 
allowed to choose from a list of topics. This raises difficulties in the 
reliability of the grading, so unless skill in choosing a topic is an ability 
being tested, this is not recommended. 

Choice of Task(s) 

Connected discourse. The task should require testees to write a piece 
of connected discourse. While there may be valid arguments for testing 
the writing of beginning students by having them just fill in blanks, once 
students are beyond the beginning stage, their writing proficiency should 
not be tested by having them translate from their native language or fill in 
blanks. 

Realistic task. The task that is chosen should reflect the type of writing 
that the testee is required to do in the real world. If the test is for students 
who will be going to English-medium universities, an appropriate task 
would be having students write an essay on an academic topic. Tasks 
which the testee would not usually perform in English would not be 
appropriate. 

Clarity. The testees should be presented with a clearly defined task 
that cannot easily be misinterpreted. Pre-testing helps insure that the 
instructions are clear and that the testees can carry out the task based on 
them. 

Modes of discourse. The test tasks should involve a mode or modes of 
discourse that are appropriate to the actual writing needs of the students. 

If necessary, the testees should be given more than one task so that they 
can demonstrate their mastery of different modes of discourse. In fact, a 
recent trend is to evaluate students, where possible, on different types of 
material that they have produced over a long period of time, rather than 
over one piece of writing on a particular occasion. 

The number of tasks. The test should involve more than one task, 
which will give an adequate sample of the testees' writing for evaluation. 
As mentioned above, different types of writing will also give a broader 
view of the students’ writing skills. 

Miscellaneous 

Level of difficulty. The teacher should carefully consider the difficulty 
of the test. Like any other test, if a writing test presents a task that is too 



easy or too difficult, if the instructions are difficult to understand, etc., the 
responses that testees give will not reflect their true ability, either because 
the task is not challenging enough for their ability or because it is so 
difficult that they do not know how to respond. Pretesting with a similar 
group is useful in determining the right level of difficulty. 

Time allowed. The teacher should carefully consider the time allowed 
for the test. If insufficient time is allowed, the students do not have a 
chance to show what they can do, and particularly in the case of assessing 
the organization of a piece of writing. However, there may be cases, such 
as writing an essays for an examination, where the task that the student 
needs to be able to do in the real world will have restrictions on time, so in 
some cases, it is appropriate to 17. allow enough writing time for a 
reasonable performance? 

Marking Tests of Writing 

The marking of writing tests will always be at least somewhat 
subjective, but the use of descriptors for each level of the marking scheme 
can at least help make the marking consistent. One possibility is to make a 
marking scheme for the overall quality of the writing, but the problem is 
that, for example, the grammar can be good but the organization poor. It 
is perhaps more useful to have different sets of descriptors for each aspect 
of writing that you want to consider. You might want to have descriptors 
for grammatical correctness, use of vocabulary, content, organization, and 
mechanics. These categories might be weighted differently, depending on 
what you want to emphasize. 



